Motion Vision and Tracking for Robots in Dynamic, Unstructured Environments

The use of robots in dynamic and unstructured environments poses difficult challenges for machine vision. The vision systems for such robots must report changes in the world quickly and must deal with objects that have not been previously seen or specified. A promising approach to building such vision systems is to construct generally applicable, modular vision services that compute summarizations of camera data that are of direct utility for action. In this thesis it is argued that such modular vision services can serve to connect machine vision to robots in dynamic, unstructured environments. The thesis describes a real-time vision service that picks out and monitors nonrigid moving objects in natural scenes. The tracking service uses correlation-based optical flow for both picking out, and monitoring the camera-relative extent of moving objects. The algorithms that comprise this service have been implemented on both massively parallel SIMD, and small-scale parallel MIMD hardware. As these algorithms are intended to be used for real-time robot systems, the algorithms and their complexity are described in detail. The tracking service has been used to build two camera-panning robot systems. Whenever a large object moves into view, the object is picked out and tracked, while the robot pans the camera to keep the object centered in the field of view. The robots have successfully tracked people walking around for long periods. These functioning robots demonstrate both the utility of the tracking vision service itself, and the manner in which such vision services can be integrated into complete systems.